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(54) Compressed picture data editing apparatus and method 



(57) An editing apparatus and the like are provided 
for splicing two data streams of compressed picture data 
at an arbitrary splice point without breaking down the 
VBV buffer. The editing apparatus inserts between a 
main stream and a sub-stream compressed by the 
MPEG-2/Video Method an inserted stream of which a 
first picture is a black I picture and succeeding pictures 
are repeat P pictures of which all macro blocks are 
skipped macro blocks. In this inserted stream, a VBV 
delay of a picture to be transmitted next, after a last pic- 
ture in the main stream, is written as a VBV delay of a 
first picture in the main stream into the picture header. 
Further, a stuffing byte of which the data amount is cal- 
culated based on the bit occupancy in the VBV buffer 
by the sub-stream at an arbitrary splice point is trans- 
mitted along with the last picture in the inserted stream. 
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Description 

[0001] The present invention relates generally to a compressed picture data editing apparatus and method and 
recording medium for storing instructions to perform such editing and to store such edited data, and more particularly 
5 to a compressed picture data editing apparatus and method in which two compressed picture data streams are spliced 
to each other at an arbitrary splice point to generate one continuous compressed picture data stream, and a recording 
medium on which the continuous compressed picture data stream is stored. 

[0002] Recently, digital transmission systems for moving pictures such as a digital broadcasting system, a visual 
telephone system, etc. have been developed. In such digital transmission systems in general, moving picture data is 

10 compressed according to the MPEG-2/Video method. MPEG is an abbreviation for the "Moving Picture coding Experts 
Group" of the ISO/IEC JTC1/SC29 (International Organization for Standardization/International Eiectrotechnical Com- 
mission, Joint Technical Committee 1/Sub Committee 29). An ISO 11172 standard is provided for an MPEG Phase 1 , 
and an ISO 13818 standard is provided for an MPEG Phase 2. In these international standards, ISO 11172-1 and 
1 3818-1 are provided for system multiplexing, IS0 11 172-2 and 1381 8-2 are provided for video coding, and IS0 11172-3 

is and 1 3818-3 are provided for audio coding. 

[0003] According to the MPEG-2/Video standard (ISO 13818-2), when decoding a compressed picture data input, 
in order to prevent an input buffer located at the front part of a decode circuit from underflowing or overflowing, the 
capacity of the input buffer should be considered, and the amount of compression during encoding should be controlled 
so that upon decoding the input buffer can accommodate the supplied data amount without underflowing or overflowing. 

20 When encoding according to the MPEG-2/Video standard, a virtual buffer for monitoring and controlling the amount of 
code that would be generated during decoding is defined as a VBV (Video Buffering Verifier) buffer, and the capacity 
of this VBV buff er is defined as VBV buffer size (vbv_buffer_size) The function of the VBV buffer will be further described 
with reference to Fig. 14, which depicts the amount of data stored in an input buffer provided upstream of a decode 
circuit when the buffer is supplied with input MPEG streams. The horizontal axis of Fig. 1 4 represents time along which 

25 decoding times (t 101 , t 10 2. fi03> •■■) for various pictures are shown while the vertical axis represents the amount of data 
(bit occupancy) stored in the input buffer. The input buffer will store the MPEG streams compressed according to the 
MPEG-2/Video Method sequentially at their respective bit rates. 

[0004] During a decoding procedure, the data corresponding to a first picture is extracted from the VBV buffer by 
the decoder at a lime t 101 . This time t 101 includes a delay (vbv_delay) from a time t 100 , the time at which the supply of 
30 the picture data is started, to allow for the data amount stored in the input buffer to reach an initial predetermined 
amount so that all of the data associated with the first few pictures can be loaded into the VBV buffer. Data representing 
the length of the delay (vbv_delay) is included as part of the MPEG stream in a picture layer. The time is represented 
by a clock of 90 kHz, for example. 

[0005] During decoding, the amount of data corresponding to a picture to be extracted from the VBV buffer by the 
35 decode circuit to be decoded is the sum of the amount of data comprising the picture (picture_size), the amount of 
data comprising a picture start code (picture_start_code), and the amount of data comprising a sequence header 
(sequence_header). The amount of data resulting from the addition of picture_size, picture__start<:ode and 
sequence_header will be referred to as "image size (image_size) n . 

[0006] After time t 101 and the extraction of the data of the first image to be decoded from the VBV buffer, the input 
40 buffer will be sequentially supplied with a succession of MPEG data streams at a predetermined bit rate corresponding 

to subsequent transmitted pictures. Also decoding of sequential pictures takes place at times t 102 , t 103 , t n , t n+1 , .... 

ADTS being a time interval between decode procedures. At each time t 102 , t 103 , .... t n , t n+1 , an amount of data 

corresponding to the image size of each picture is extracted by the decode circuit from the VBV buffer for decoding. 

Thus, the input buffer provided upstream of the decode circuit will store the MPEG streams received sequentially at 
45 their respective bit rates while an amount of data corresponding to image size of each picture is extracted by the decode 

circuit at each of the decoding times (t l01) t-, 02 , t 103 , ...). 

[0007] If the total amount ol data of the supplied MPEG streams exceeds the total amount of data corresponding to 
the image sizes of the pictures extracted from the VBV buffer at the decoding times, that is more data is added to the 
VBV buffer than is removed from it, the input buffer will eventually overflow when this excess data exceeds the size of 

so the VBV buffer. If the total amount of data of the image sizes of the pictures extracted for decoding exceeds the total 
amount of data of the supplied MPEG streams, that is less data is added to the VBV buffer than is removed from it, 
the input buffer will underflow because more data will be taken out of the VBV buffer than will be input to it. To avoid 
overflowing or underflowing the VBV buffer, the MPEG-2/Video (ISO 13818-2) standard prescribes that when encoding 
MPEG streams, the capacity of an input buffer provided upstream of a decode circuit during decoding should be mod- 

55 eled as a VBV buffer size (vbv_buffer_size) to allow for control of the amount of encoded data produced during encoding. 
The model input buffer is thus designed to accommodate an appropriate amount of data during encoding so that during 
decoding, the amount of data contained in the VBV buffer will remain within a predetermined range. 
[0008] Generally, at a broadcasting station or the like, in order to transmit a moving picture sequence that has been 
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compressed according to the MPEG-2 system, often two or more sets of moving picture data are edited and spliced 
together to form one set of moving picture data. For example, at the broadcasting station, moving picture data of a 
movie may be edited to insert a commercial into the movie as shown in Fig. 15. Each of the commercials includes an 
amount of data that allows the commercial to run for a short period of time. When conventionally performing such an 
5 edit, for real-time transmission and for convenience in editing, base-band moving picture data are not handled during 
editing. Rather, compressed MPEG streams are processed during editing without decoding. 

[0009] First to third conventional alternatives for performing such editing will now be described. In each of the con- 
ventional alternatives, a first picture data stream compressed according to the MPEG-2/Video Method ("main stream" 
hereinafter) has spliced thereto a second picture data stream also compressed according to the MPEG-2/Video Method 
10 ("sub-stream" hereinafter) to generate one spliced data stream. The conventional first to third alternatives will be de- 
scribed with reference to Figs. 16 to 18, which show the amount of data in the VBV buffer attributable to the main 
stream, the amount of data in the VBV buffer attributable to the sub-stream, and the amount of data in the VBV buffer 
attributable to the spliced stream, respectively. 

[0010] First, referring to Fig. 16, the conventional first alternative will be described. As is shown in Fig. 16A, a splice- 
rs out point at which the main stream is to be spliced out is defined at as a time (t ni ). At this splice-out point, the supply 
of data to the VBV buffer from the main stream will end. In the first conventional alternative, a splice-in point of the 
sub-stream is defined at a time (t 121 ). At this splice-in point the supply of the sub-stream to the VBV buffer will start, 
as shown in Fig. 16B. According to the first conventional alternative, the splice-out and splice-in points of the main 
stream and sub-stream, respectively, are spliced to each other at the time x m so that the picture in the main stream 
20 js extracted by a decode circuit until a time t 112 and the picture in the sub-stream is extracted by the decode circuit 
after the time t 112 to generate one spliced stream, as shown in Fig. 16C, for example. In the case of the spliced stream 
thus generated according to the first conventional alternative, however, the difference between the total data amount 
of the supplied picture and the total data amount of the extracted picture may exceed the VBV buffer size so that the 
VBV buffer will possibly overflow as indicated at the hatched portion in Fig. 16CH ; 
2S [0011] Next, the second conventional alternative will be described, making reference to Fig. 17. As is shown in Fig. 
17 A, a splice-out point of the main stream is defined at a time (t 131 ) at which the last picture from the mainstream will 
be extracted by the decode circuit. In this second conventional alternative, a splice-in point of the sub-stream is defined 
at a time (t 141 ) at which time the amount of data contained in the VBV buffer will be the same as the amount of data 
contained in the VBV buffer at the splice-out point of the main stream, as shown in Fig. 1 7B. In the second conventional 
30 alternative, the splice points of the main stream and sub-stream, respectively, are spliced to each other at the time t 131 
so that the data associated with the picture of the main stream is extracted from the buffer by a decode circuit until a 
time t 13l , and thereafter the data associated with the picture in the sub-stream is extracted by the decode circuit to 
generate one spliced stream, as shown in Fig. 17C, for example. In this second conventional alternative, however, 
since the first picture in the sub-stream is extracted at a time at which a next picture would have been extracted from 
35 the main stream, if provided (next_vbv_delay), the total data amount of the extracted picture datafrom the VBV buffer 
may be larger than the total amount of data that has been supplied to the VBV buffer as shown in Fig. 17C. The VBV 
buffer may underflow. More particularly, since the VBV delay of the next picture after the last picture to be extracted 
from the main stream (next_vbv_deiay) is shorter than the delay of the first picture to be extracted from the sub-stream 
(first_vbv_delay), the VBV buffer may underflow. 
40 [0012] Finally, the third conventional alternative will now be described with reference to Fig 1 8. As shown in Fig. 18A, 
a splice-out point of the main stream is taken as a time (t 151 ) at which a last picture will be extracted from the VBV 
buffer by the decode circuit and the amount of data in the VBV buffer is zero. In this third conventional alternative, a 
splice-in point of the sub-stream is taken as a time (t 161 ) at which time the supply of the sub-stream data to the VBV 
buffer will start, as shown in Fig. 18B. The splice points of the main stream and sub-stream are spliced to each other 
45 at the time t 151 so that the picture supplied from the main stream is extracted from the VBV buffer by a decode circuit 
until a time t 151 . Thereafter, the picture supplied from the sub-stream is extracted by the decode circuit after the time 
t 152 to generate one spliced stream, as seen from Fig. 18C, for example. In this third conventional alternative, since 
the supply of the sub-stream data starts after the last picture supplied by the main stream has been extracted from the 
VBV buffer, the VBV buffer will not overflow or underflow. However, there is a time zone (between t 151 and t 152 ) in 
50 which the decode circuit cannot extract any picture from the VBV buffer. Similarly, no data is supplied to the VBV buffer 
from the time (t 150 ) at which the last picture from the main stream data is supplied until the time (t 151 ) at which the 
supply of the sub-stream data starts. Therefore, the third conventional alternative does not allow for continuous pro- 
vision of MPEG stream data to the decoder. The decoder is unable to properly process the MPEG stream, and as a 
result does not allow for stable decoding of MPEG picture data. 
55 [0013] Therefore, it would be beneficial to provide an editing apparatus and method that addresses the drawbacks 
of the prior art. 

[0014] Still other objects and advantages of the invention will in part be obvious and will in part be apparent from the 
specification and the drawings. 
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[0015] In accordance with at least preferred embodiments of the invention, a compresse picture data editing appa- 
ratus is provided for generating a continuous compressed picture data by splicing an arbitrary splice point of a first 
compressed picture data stream (first elementary stream) and an arbitrary splice point of a second compressed picture 
data stream (second elementary stream). The first and second elementary streams are spliced by inserting between 

s the arbitrary splice point of the first elementary stream and the splice point of the second elementary stream a com- 
pressed picture data inserted stream including a predetermined compressed picture data and a pseudo data. A bit 
occupancy in a virtual buffer is determined for the first elementary stream at the arbitrary splice point, which controls 
an amount of code generation during encoding. A bit occupancy is also determined in the virtual buffer for the second 
elementary stream at the arbitrary splice point, and for an elementary inserted stream. The splicer is controlled based 

io on the detected bit occupancies and the amount of compressed picture data in the elementary inserted stream and 
the amount of required pseudo data to make the bit occupancy of the elementary inserted stream in the first virtual 
buffer equal to the amount of compressed picture data in the first elementary stream in the virtual buffer at the arbitrary 
splice point. Additionally, the bit occupancy in the virtual buffer determined by the amount of compressed picture data 
j n the elementary inserted stream is made to coincide with that in the second elementary stream in the virtual buffer 

15 at the arbitrary splice point. 

[0016] As a feature of at least preferred embodiments of the present invention, the bit occupancy in the first virtual 
buffer for an elementary inserted stream is made to coincide with the bit occupancy in the first virtual buffer for the first 
elementary stream at the arbitrary splice point. The bit occupancy in the last virtual buffer attributable to the elementary 
inserted stream is similarly made to coincide with the bit occupancy in the virtual buffer for the second elementary 

20 stream at the arbitrary splice point. The elementary stream is inserted between the first and second elementary streams 
to generate a continuous compressed picture data stream. 

[0017] As another feature of at least preferred embodiments of the present invention, when the bit occupancy in the 
virtual buffer by the elementary inserted stream exceeds the size of the virtual buffer, a pseudo data larger than the 
excess data for which the bit occupancy exceeds the virtual buffer size, is inserted between the first and second ele- 
25 mentary streams so that the actual buffer utilized during decoding will not overflow. 

[0018] According to the at least preferred embodiments of the present invention the compressed picture data and 
an elementary inserted stream, in which the pseudo data is inserted, are inserted between the first and second ele- 
mentary streams to generate a continuous compressed picture data stream that will not overflow or underflow a buffer 
utilized during decoding. 

30 [0019] The invention accordingly comprises the several steps and the relation of one or more of such steps with 
respect to each of the others, and the apparatus embodying features of construction, combinations of elements and 
arrangement of parts which are adapted to effect such steps, all as exemplified in the following detailed disclosure, 
and the scope of the invention will be indicated in the claims. 

[0020] An embodiment of the invention will now be described, by way of example only, with reference to the accom- 
35 panying drawings in which: 

Fig. 1 depicts an editing apparatus constructed in accordance with one embodiment of the invention; 
Fig. 2 depicts the editing apparatus of Fig. 1 in greater detail; 

Fig. 3 is a block diagram depicting the structure of editing apparatus according to an example embodiment of the 
40 invention. 

Fig. 4 shows a transition of the bit occupancy in a VBV buffer in accordance with a main stream supplied to the 
editing apparatus of an example embodiment of the invention. 

Fig. 5 shows a transition of the bit occupancy in the VBV buffer in accordance with a sub-stream supplied to the 
editing apparatus of an example embodiment of the invention; 
45 Fig. 6 shows a transition of the bit occupancy in the VBV buffer in accordance with a spliced stream generated by 

the editing apparatus of an example embodiment of the invention; 

Fig. 7 shows a transition of the bit occupancy in the VBV buffer at a splice point between a main stream and an 

inserted stream in a spliced stream generated by the editing apparatus of an example embodiment of the invention; 

Fig. 8 shows a transition of the bit occupancy in the VBV buffer in accordance with an inserted stream generated 
so by the editing apparatus of an example embodiment of the invention; 

Fig. 9 shows a transition of the bit occupancy in the VBV buffer at a splice point between an inserted stream in a 

spliced stream generated by the editing apparatus of an example embodiment of the invention and a sub-stream; 

Fig. 10 shows a transition of the bit occupancy in the VBV buffer in accordance with a spliced stream generated 

by the editing apparatus of an example embodiment of the invention; 
55 Fig. 1 1 is a flow chart depicting the operation of the editing apparatus in accordance with an example embodiment 

of the invention; 

Fig. 12 is a further flow chart depicting operation of the editing apparatus in accordance with an example embod- 
iment of the invention; 
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Fig. 1 3 is a drawing depicting operations by which commercials are inserted into a continuous picture data such 
as a movie; 

Fig. 14 illustrates a transition ol the bit occupancy by an MPEG stream in an input buffer provided upstream of a 
decoder when the input buffer is supplied with the MPEG stream; 
5 Fig. 15 is an illustration depicting commercials being inserted in a movie to produce a program; 

Figs. 16A to 16C show a bit occupancy by a main stream, a sub-stream and a spliced stream in a VBV buffer in 
a conventional apparatus, respectively; 

Figs. 17A to 17C show a bit occupancy by a main stream, a sub-stream and a spliced stream in a VBV buffer in 
a conventional apparatus, respectively; and 
fo Figs. 18A to 18C also show a bit occupancy by a main stream, a sub-stream and a spliced stream apparatus, in 

a VBV buffer in a conventional apparatus, respectively. 

[0021] An editing apparatus constructed in accordance with the invention is adapted to edit an MPEG-2/Video-com- 
pressed picture data by splicing to the trailing end of an arbitrary picture in a data stream (first elementary stream) of 
*5 a first picture data compressed by the MPEG-2/Video Method a data stream (second elementary stream) of a second 
picture data compressed by the MPEG-2/Video Method to generate a spliced stream. 

[0022] Referring first to Fig. 1 , an editing apparatus 1 constructed in accordance with the invention is utilized, for 
example, for inserting in a timely manner, a second elementary stream (will be referred to as "sub-stream" hereinafter) 
such as commercials or the like, each of a short duration and compressed by the MPEG-2/Video Method, into a first 

20 elementary stream (will be referred to as "main data stream" hereinafter) such as a movie or the like compressed by 
the MPEG-2/Video Method to produce a single program data stream ("spliced stream"). The spliced stream may be 
recorded on a recording medium such as a disc or tape, or may alternatively be transmitted to a receiving terminal via 
a transmission line such as a satellite, broadcasting network, standard terrestrial wave transfer line or Internet. 
[0023] As is shown in Fig. 2, editing apparatus 1 is employed in a local broadcasting station to receive a main stream, 

2S including a movie, news or a similar program transmitted from a main, possibly remote broadcasting station. Editing 
apparatus 1 then generates a spliced stream or local program by inserting into the main stream of the movie or the 
news program a sub-stream such as a local weather-forecast, local commercial stored in a server, on-the-spot live 
image supplied from a live, local camera or the like, and the spliced stream is transmitted to a receiving terminal via a 
transmission line. 

30 [0024] Fig. 3 depicts a block diagram of editing apparatus 1 constructed in accordance with the invention. Editing 
apparatus 1 is supplied with a main data stream whose data ends at an arbitrary picture, and a sub-stream whose data 
starts at an arbitrary picture from a server or the like (not illustrated). Arbitrary pictures are selected by a user from the 
main data stream and sub-stream in accordance with the operation of editing apparatus 1 during an editing process. 
These arbitrary pictures include a last picture in the main data stream and a first picture in the sub-stream. A main data 

35 stream splice-out point and sub-stream splice-in point are determined based upon these user selected pictures. The 
main data stream and the sub-stream are spliced together at the splice points by editing apparatus 1 to generate a 
spliced stream. 

[0025] As is shown in Fig. 3, editing apparatus 1 comprises a picture decode time detector 2 to detect a VBV delay 
of a picture to be transmitted next after a last picture in the main data stream, a VBV delay detector 3 to detect a VBV 

40 delay of a first picture in a sub-stream, a black I picture server 4 having stored therein black I pictures, a repeat P 
picture server 5 having stored therein repeat P pictures, a picture select switch 6 to select between the black I picture 
and the repeat P picture, a stuffing byte generator 7 to generate a stuffing byte, a multiplexer 8 to receive the main 
data stream, sub-stream, block I picture or repeat P picture and stuffing byte and selecting one of them to generate a 
spliced stream, and a controller 9 to control all of these components. 

45 [0026] Picture decode time detector 2 is supplied with a main data stream whose data ends at an arbitrary, user 
selected picture, namely, at a splice-out point. Picture decode time detector 2 detects a VBV delay of a picture to be 
transmitted as the picture next after this last picture. That is, picture decode time detector 2 detects a VBV delay of 
this next picture of the sub-stream that is to be spliced to the splice-out point of the main stream. A VBV delay 
(next_vbv_delay) of a picture to be transmitted after a selected arbitrary predetermined picture can be determined by 

so calculating the following relation (1), from a VBV delay (vbv_delay) of the predetermined picture, an image size 
(image_size) of the predetermined picture, and a bit rate (bitrate) of a data stream to be transmitted. 

next_vbv_delay = vbv_delay - (image„size/bitrate) + ADTS (1 ) 

55 

Therefore, in the context of the invention, the VBV delay (next_vbv_delay) of the picture to be transmitted next to, and 
immediately after the last picture in the main data stream can be determined, based on the following relation (2), from 
a VBV delay (last_vbv_delay) of the last picture in the main data stream, an image size (image_size) of the last picture, 
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an interval of decode time stamp ADTS, and a bit rate (bitrate) of the main stream: 



next_vbv_delay = last_vbv_delay - (image_size/bitrate) + ADTS (2) 

5 

[0027] Fig 4 shows the transition of the amount of data from the main data stream stored in the VBV buffer over 
time and also depicts the values of the parameters in relation (2). The picture decode time decoder 2 supplies the 
controller 9 with the VBV delay (next_vbv_delay) of a picture to be decoded next after the last picture in the main 
stream, having been thus determined. 

10 [0028] VBV delay detector 3 of Fig. 2 is supplied with a sub-stream whose data starts at an arbitrary picture. VBV 
delay detector 3 detects a VBV delay (first_vbv_delay) of the first picture in the sub-stream. Fig. 5 shows the transition 
of the amount of data from the sub-stream stored in the VBV buffer, and also the VBV delay (first_vbv_delay) of the 
first picture in the sub-stream. VBV delay detector 3 supplies controller 9 with the detected VBV delay (first_vbv_delay) 
of the first picture in the sub-stream. 

is [0029] Black I picture server 4 of Fig. 2 has stored therein a picture data of a black I picture and outputs the black I 
picture under the control of controller 9. Repeat P picture server 5 has stored therein a picture data of a P picture (will 
be referred to as "repeat P picture" hereinafter) of which each macro block is composed of a skipped macro block. In 
the MPEG-2, a macro block requiring no information and unencoded is called a skipped macro block (skipped MB). 
When the skipped macro block is decoded, the information in a preceding macro block will be repeated. In the repeat 

20 P picture, each macro block is composed of such a skipped macro block. Repeat P picture server 5 outputs a stored 
repeat P picture under the control of controller 9, thus repeating the display of the previously displayed picture. Under 
the control of controller 9, picture select switch 6 selects either a black I picture or a repeat P picture output from black 
I picture server 4 or repeat P picture server 5, respectively. 

[0030] Stuffing byte generator 7 generates a stuffing byte (stuffing_byte) as a pseudo data produced when the amount 
25 of code data generated in the coding of an MPEG stream is small. The stuffing byte has no special meaning and will 
be discarded at the decoder. The code amount and output timing of the stuffing byte are controlled by controller 9. 
Controller 9 controls and monitors all the above-mentioned components of editing apparatus 1 , and controls the switch- 
ing of multiplexer 8 and also of picture select switch 6. 

[0031] Multiplexer 8 is supplied with the main data stream, sub-stream, black I picture or repeat P picture, and stuffing 
30 byte, and selects one of these inputs to generate a spliced stream under the control of controller 9. More particularly, 
multiplexer 8 inserts between the splice-out point of the main stream and the splice-in point of the sub-stream a data 
stream (will be referred to as "inserted stream- hereinafter) comprised of a compressed picture data including a black 
I picture, repeat P picture, stuffing byte, etc. as shown in Fig. 6. 

[0032] Next, the inserted stream to be inserted between the main data stream and a sub-stream wilt now be described. 

35 As is shown in Fig. 7, controller 9 switches multiplexer 8 upon transmission of the last picture in the main data stream, 
namely, the data up until the splice-out point. Thereafter, the transmission of data is continued by a transmission of a 
black I picture stored in black I picture server 4 as a first part of an inserted stream. At this time, the amount of data 
(first_picture_occupancy) of the black I picture to be stored in the VBV buffer is determined based on a VBV delay 
(next_vbv_delay) of a picture to be transmitted next, after the last picture in the main stream, the delay having been 

40 detected by the picture decode time detector 2, and written into the picture header. 

[0033] The amount of data of each picture is determined by calculating the following relation (3): 



45 



Amount of data = vbv_delay x bitrate/lrequency ( 3 ) 



where the "vbv_delay" is a VBV delay of the picture in consideration, and "frequency" is a clock frequency for indication 
of the VBV delay. The "frequency" is represented by the unit of 90 kHz, for example. 
[0034] With the above operations, the amount of data (first_picture_occupancy) available for storing the black I picture 
is determined, by calculating the following relation (4), from the VBV delay (next_vbv_delay) of a picture to be decoded 
50 next, after the last picture in the main data stream, that has been detected by picture decode time detector 2. 

first_picture_occupancy = next_vbv_delay x bitrate/frequency (4) 

55 Editing apparatus 1 can splice the main data stream and inserted stream continuously by writing an amount of data of 
the picture to be decoded after the next to last picture in the main data stream (corresponding to the VBV delay 
(next_vbv_delay). This amount of data is referred to as the (first_picture_occupancy) of the first picture in the inserted 
stream. The spliced stream thus generated will not break down the input buffer and can be decoded without any 
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restrictions imposed upon the operations of editing apparatus 1. 

[0035] Upon completion of the transmission of the black I picture, controller 9 switches picture select switch 6 for 
transmission of repeat P pictures stored in repeat P picture server 5. Controller 9 will transmit a preset number of repeat 
P pictures in turn, one after another in sequence following the black I picture. Controller 9 will therefore determine 

5 preset time intervals between a main stream and sub-stream to be spliced to each other, determine a display time and 
transmission time for the inserted stream and transmit a predetermined number of repeat P pictures corresponding 
to the determined time interval. In transmitting each repeat P picture, controller 9 monitors operation so that the bit 
occupancy will not exceed the buffer size (vbv_buffer_size) so that the inserted stream will not break down the VBV 
buffer. If the inserted stream overflows the VBV buffer, controller 9 will switch multiplexer 8 to transmit a stuffing byte 

10 generated by the stuffing byte generator 7 following each repeat P picture, thereby reducing the amount of data that 
will eventually be stored in the decoding buffer. 

[0036] More particularly, the data amount of the stuffing byte can be determined as shown in Fig. 8 and as will be 
described below. First, controller 9 determines a VBV delay (next_vbv_delay) of a next repeat P picture to be transmitted 
from a VBV delay (vbv_delay) and image size (image_size) of each repeat P picture, according to the equation 

is 

next_vbv_delay = vbv_delay - (image_size/bitrate)+ADTS 

Controller 9 determines a bit occupancy (next_picture_occupancy) of a next repeat P picture to be transmitted from 
20 the determined VBV delay (next_vbv_delay) in accordance with the equation 

next_picture_occupancy = next_vbv_delay x bitrate/frequency 

25 The size of a stuffing byte to be inserted into each picture can be determined from a difference between the bit occu- 
pancy (next_picture_occupancy) of a next repeat P picture and the VBV buffer size (vbv_buffer_size) in accordance 
with the equation 

3o stuffing-byte = vbv_buffer_size - next_picture_occupancy 

When next_vbv_occupancy is 55,048, bitrate is 3,000,000 BPS, vbv_buffer_size is 1 ,835,008 bits and frequency is 90 
kHz, for example, the data amount of the stuffing byte can be determined by calculating the following relation: 

35 next_picture_cccupancy = 55048 x 3000000/90000 = 1 834933 



stuff ing_byte = 1835008 - 1834933 = 75 bit = 9.4 byte 

40 

By repeating the insertion of a repeat P picture and a stuffing byte sequentially, thus generating a continuous inserted 
stream as an MPEG data stream while monitoring the inserted stream and VBV buffer size, editing apparatus 1 can 
generate a spliced stream without breaking down the VBV buffer. 

[0037] Further, upon completion of the transmission of a predetermined number of repeat P pictures, or upon com- 
45 pietion of the transmission of data to a splice-out point of the inserted stream, controller 9 will switch the multiplexer 8 
and start transmission of the sub-stream as shown in Fig. 9. At this time, controller 9 will adjust the data size of the 
stuffing byte to be inserted into the last picture in the inserted stream based on the VBV delay (first__vbv_delay) of the 
first picture in the sub-stream, this VBV delay of the first picture having been detected by VBV delay detector 3. In this 
manner, the inserted stream and sub-stream may be spliced to each other. 
so [0038] More particularly, controller 9 determines the data size of the stuffing byte to be inserted into the last picture 
in the inserted stream as will now be described. First, controller 9 determines the decode size of the first picture of the 
sub-stream (first_picture_decode_size) at the splice point between the inserted stream and the first picture of the sub- 
stream in accordance with the following equation: 

55 first_picture_decode_size = (first_vbv_delay - ADTS) x bitrate + picture_start_ code + sequence_ header, 

where the picture_start__code and the sequence_header are those of the first picture in the sub-stream. Then, controller 
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9 calculates a stuffing byte (stuffing_byte) to be inserted into the last picture in the inserted stream from the decode 
size of the first picture of the sub-stream (first_picture_decode_size) and a bit occupancy (last_picture_occupancy) of 
the last picture in the inserted stream in accordance with the equation. 



stuffing_byte = last_picture_occupancy - first_picture_decode_size. 



Editing apparatus 1 splices the inserted stream and the sub-stream continuously to each other by inserting the stuffing 
byte thus determined following the last picture in the inserted stream to generate a spliced stream. The continuous 
io spliced stream can be decoded without breaking down the input buffer and without any restrictions on the operations 
of editing apparatus 1 . 

[0039] While the format of the inserted stream to be inserted between the main stream and sub-stream has been 
described above, in an alternative embodiment, the inserted stream may have the following format. In the inserted 
stream, the top I picture is not limited only to a black picture. Namely, the first image in the inserted stream may be 

is any picture which is not limited in color and content. For example, when a black picture is inserted at the front of the 
inserted stream, it will be displayed between the main stream and sub-stream on a monitor. However, a blue picture, 
green picture or a predetermined comment or still image may be displayed between the main stream and sub-stream. 
Also, the I picture need not be inserted at the front of the inserted stream, but a repeat P picture may be inserted at 
the front. In this case, the last image in the main stream will be displayed as a frozen image. Further, the repeat P 

20 pictures in the inserted stream may not be limited to a fixed number but the number of the repeat P pictures may be 
variable depending upon the relation between the main stream and sub-stream. As shown in Fig. 10, for example, the 
decode size (first_picture_decode_size) of a splice point between the inserted stream and sub-stream may be deter- 
mined as shown above. Similarly, the decode size (decode_size) of the splice point between the main stream and sub- 
stream may also be determined. When one repeat P picture is inserted, a predetermined bit occupancy will be added, 

25 and thus a minimum necessary number of repeat P pictures to be inserted is calculated based on a difference between 
the decode size (first_picture_decode_size) of the splice point between the inserted stream and sub-stream and the 
decode size (decode_size) of the splice point between the main stream and sub-stream. In this case, the number of 
repeat P pictures included in the inserted stream can be set to a minimum required number. 

[0040] Next, the operation of editing apparatus 1 will be described making reference to the flow charts in Figs. 11 

30 and 12. First at Step S1 shown in Fig. 11, when a main stream is supplied to editing apparatus 1, a VBV delay 
(next_vbv_delay) of a picture to be transmitted next, after the last picture in the main stream is determined as noted 
above. At Step S2, it is determined whether an I picture or repeat P picture is inserted as the first picture of an inserted 
stream. For insertion of the I picture, the operation proceeds to Step S3. For inserting of the repeat P picture, the 
operation proceeds to Step S4. This selection may be performed by a user. 

35 [0041] At Step S3, an I picture being a black picture, for example, is inserted as the first picture in the inserted stream. 
Then a bit occupancy (first_picture_occupancy) of the black I picture is determined as noted above, and is written into 
the picture header. Also, if the bit occupancy (next_picture_occupancy) of a next repeat P picture exceeds the VBV 
buffer size (vbv_buffer_size), the size of a stuffing byte (stuffing_byte) is determined as noted above and is inserted 
into the first picture in the inserted stream. The operation then proceeds from Step S3 to Step S4. At Step S4, it is 

40 determined whether the number of repeat P pictures to be inserted into the inserted stream is a fixed number, or is a 
minimum necessary number of pictures to be determined. If the number of repeat P pictures is to be fixed, the operation 
proceeds to Step S5. When the number of repeat P pictures is to be a minimum number of pictures to be determined, 
the operation proceeds to Step 10 shown in Fig. 12. This selection may be performed by a user. 
[0042] At Step S5, a repeat P picture is inserted. Next at Step S6, the bit occupancy (next_picture_occupancy) of a 

45 next repeat P picture is calculated, and at next Step S7, it is d* -ermined whether the bit occupancy 
(nexl_picture_occupancy) of a next repeat P picture exceeds the VBV buffer si>:e (vbv_buff er_size). If the bit occupancy 
exceeds the VBV buffer size, the size of a stuffing byte (stuffing_byte) is determined at Step S8 as noted above and 
is inserted. Then, operation proceeds to Step S9. If the determined bit occupancy does not exceed the VBV buffer 
size, the operation proceeds directly to Step S9. 

so [0043] At next Step S9, it is determined whether a set number of repeat F ^ctures has completely been inserted. If 
the determination is answered in the affirmative, operation proceeds to Step S16. If the determination is answered in 
the negative, the operation returns to Step S5 and the procedure from Step S5 is repeated. 

[0044] On the other hand, if it is determined at step S4 that the number of repeat P pictures is to be a calculated 
minimum number, operation proceeds to step S10. At Step S10 shown in Fig. 12, the decode size (first_picture_size) 
55 of the splice point between the inserted stream and sub-stream, and the decode size (decode_size) of the splice point 
between the main stream and sub-stream, are determined. Further, at Step S11 , a repeat P picture is inserted, and at 
Step S12, the bit occupancy (next_picture_occupancy) of a next repeat P picture is calculated. 
[0045] At Step SI 3, it is determined whether the bit occupancy (next_picture_occupancy) of a next repeat P picture 
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exceeds the VBV buffer size (vbv_buffer_size). If the bit occupancy of the next P picture is determined to exceed, the 
VBV buffer size, the size of a stuffing byte (stuffing_byte) is determined at Step S14 as noted above, and is inserted. 
Operation then proceeds to Step S15. If the bit occupancy of the next repeat P picture is determined not to exceed the 
VBV buffer size, the operation proceeds directly to Step S15. 
5 [0046] At Step S15, it is determined whether the current decode size exceeds the decode size 
(first_picture_decode_size) of the splice point between the inserted stream and sub-stream. If the determination is 
answered in the affirmative, operation proceeds to Step S16 shown in Fig. 11. If the determination is answered in the 
negative, operation is returned to step S10, and the procedure from Step S10 is repeated. 

[0047] At Step S1 6 shown in Fig. 1 1 , the VBV delay (first_vbv_delay) of the first picture in the sub-stream is detected, 
10 and at Step S17, the size of the stuffing byte (stuffing_byte) to be inserted into the last picture in the inserted stream 
is determined as noted above, and the stuffing byte is inserted. The inserted stream and sub-stream are then spliced 
to each other at the splice point and the operation is completed. 

[0048] By conducting the operations included in Steps S1 to S17, the editing apparatus 1 inserts the inserted stream 
between the main stream and sub-stream to generate a continuous spliced stream. As described, the embodiment of 

is the editing apparatus according to the present invention generates an inserted stream in which the bit occupancy of 
the inserted stream in the first VBV buffer coincides with the bit occupancy in the VBV buffer ol the main stream at the 
splice point, and the bit occupancy of the inserted stream in the last VBV buffer coincides with the bit occupancy in the 
VBV buffer of the sub-stream at an arbitrary splice point. Thus, editing apparatus 1 can splice the main stream and 
sub-stream continuously to each other without breaking down the VBV buffer. A spliced stream generated through 

20 splicing of two streams can also be decoded as usual without breaking down the buffer provided at the input stage in 
the decoder, namely, with no overflow and underflow, as well as with no restrictions imposed upon the editing operations. 
[0049] It should be noted that the editing apparatus 1 according to the present invention can insert commercials into 
a part of a succession of moving pictures like a movie as shown in Fig. 1 3, for example, by swapping the main stream 
and sub-stream input terminals with each other. First, a former half of the picture data such as a movie is taken as the 

25 main stream while the commercials are taken as sub-streams. Thus, a commercial is spliced to the former half of the 
movie. Then, the commercial spliced to the former half of the movie is taken as the main stream while the later half of 
the movie is taken as the sub-stream. The latter half of the movie is spliced to the commercial. Thus, the editing 
apparatus 1 can insert commercials into a part of a succession of moving pictures like a movie. 
[0050] Furthermore, the invention may be realized as a computer program performed in a general purpose computer, 

30 having input to it a main stream and a sub-stream, and outputting a combined stream. 

[0051] The compressed picture data editing apparatus and method according to the present invention is adapted to 
j nser t between first and second elementary streams an inserted stream so that a bit occupancy in a virtual buffer for 
controlling a first amount of code generation coincides with a bit occupancy in the virtual buffer by the first elementary 
stream at an arbitrary splice point and a bit occupancy in a virtual buffer for controlling a last amount of code generation 

35 coincides with a bit occupancy in the virtual buffer by the second elementary stream at an arbitrary splice point, to 
thereby generate a continuous compressed picture data stream. A recording medium in accordance with the invention 
is adapted to store the continuous compressed picture data stream. Thus, the compressed picture data editing appa- 
ratus, method and recording medium according to the present invention can splice two data streams of compressed 
picture data directly to each other without breaking down the virtual buffer for controlling the amount of code generation. 

40 Therefore, an elementary stream generated through splicing of the two compressed picture data to each other can be 
decoded as usual without breaking the buffer provided at the input stage, namely, without overflowing and underflowing 
as well as with no restrictions imposed on the editing operations. 

[0052] Also, the apparatus and method according to the present invention can insert between first and second ele- 
mentary streams^ an elementary inserted stream which will not break down the virtual buffer for controlling the amount 
45 of code generation, by inserting between the first and second elementary streams, the elementary inserted stream in 
which a compressed picture data and a pseudo data are inserted. 

[0053] Additionally, the apparatus and method according to the present invention permit the display of a still image 
of an arbitrary splice point of the first compressed picture data and also display of an arbitrary image such as a black 
picture or the like, while an inserted elementary inserted stream is inserted between the splice points of the first and 
so second elementary streams, respectively, by inserting a coded picture, repeat picture or the like as the elementary 

inserted stream. 

[0054] It will thus be seen that the objects set forth above, among those made apparent from the preceding descrip- 
tion, are efficiently attained and, since certain changes may be made in carrying out the above method and in the 
constructions set forth without departing from the scope of the invention, it is intended that all matter contained in the 
55 above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting 
sense. 

[0055] At least preferred embodiments of the invention provide: 
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an improved editing apparatus and method that overcomes the above-mentioned drawbacks of the prior art; 
an improved compressed picture data editing apparatus and method adapted to generate one continuous com- 
pressed picture data stream by splicing two data streams of compressed picture data to each other at an arbitrary 
splice point in such a manner that an input buffer of a decoder will not overflow, underflow, or otherwise break 
down and in which the decoder is not constrained in its ability to process the spliced data stream; and 
a recording medium for storing a continuous compressed picture data stream generated according to the invention. 

[0056] It is also to be understood that the following claims are intended to cover all of the generic and specific features 
of the invention herein described and all statements of the scope of the invention which, as a matter of language, might 
be said to fall therebetween. 



Claims 

1. A method for splicing a first elementary data stream containing first compressed picture data and a second ele- 
mentary data stream containing a second compressed picture data to generate a spliced elementary data stream 
containing a spliced continuous compressed picture data, said spliced elementary data stream being generated 
by inserting an inserted elementary data stream between an arbitrary splice point ol the first elementary data 
stream and an arbitrary splice point of the second elementary data stream, said method comprising the steps of; 

providing a virtual buffer for controlling an amount of code data which is generated; 

detecting a bit occupancy in said virtual buffer at an arbitrary splice point of said first elementary data stream; 
detecting a bit occupancy in said virtual buffer at an arbitrary splice point of said second elementary data 
stream; and 

controlling said inserted elementary data stream so that a first bit occupancy of the inserted elementary data 
stream in said virtual buffer equals the bit occupancy in said virtual buffer at the arbitrary splice point of the 
first elementary data stream and a last bit occupancy of the inserted elementary data stream in said virtual 
buffer equals the bit occupancy in said virtual buffer at the arbitrary splice point of the second elementary data 
stream. 

2. The method according to claim 1 , further comprising the step of: 

generating the spliced elementary data stream by inserting the inserted elementary data stream between 
the arbitrary splice point of the first elementary data stream and the arbitrary splice point of the second elementary 
data stream. 

3. The method according to claim 1 , wherein the inserted elementary data stream comprises a predetermined com- 
pressed picture data and pseudo data. 

4. The method according to claim 3, wherein the virtual buffer exhibits a predetermined size; and further comprising 
the steps of: 

determining if the bit occupancy of the inserted elementary data stream in the virtual buffer exceeds the pre- 
determined size of the virtual buffer; and 

controlling the pseudo data to have a size larger than the data amount that the inserted elementary data stream 
exceeds the predetermined size of the virtual buffer. 

5. The method according to claim 3, wherein the inserted elementary data stream includes at least one intra-coded 
picture. 

6. The method according to claim 5, wherein the at least one intra-coded picture is a black picture. 

7. The method according to claim 5, wherein the inserted elementary data stream includes at least one repeat picture 
following the intra-coded picture, the repeat picture repeatedly displaying a proceeding picture. 

8. The method according to claim 7, further comprising the step of inserting at least one repeat picture until the last 
bit occupancy ol the inserted elementary data stream in the virtual buffer exceeds the bit occupancy in the virtual 
buffer at the arbitrary splice point of the second elementary data stream; and 

inserting pseudo data into the spliced elementary data stream thereafter 
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The method according to claim 3, wherein the inserted elementary data stream includes a number of repeat pic- 
tures, the repeat picture repeatedly displaying a previously displayed picture. 

0. The method according to claim 9, further comprising the step of inserting at least one repeat picture until the last 
bit occupancy of the inserted elementary data stream in the virtual buffer exceeds the bit occupancy in the virtual 
buffer at the arbitrary splice point of the second elementary data stream; and 

inserting pseudo data into the spliced elementary data stream thereafter. 

1. The method according to claim 1 , wherein the first and second elementary data streams and the inserted elemen- 
tary data stream are coded data according to MPEG video format, respectively 

2. The method according to claim 11, wherein the inserted elementary data stream comprises a predetermined coded 
picture data coded by MPEG video format and a stuffing byte defined in MPEG video format. 

3. The method according to claim 1 2, further comprising the steps of: 

detecting the bit occupancy in the virtual buffer (VBV buffer) by the following equation: 

bit occupancy = VBV_delay x (bitrate/frequency) 

wherein VBV_delay represents a VBV delay of a picture in an elementary data stream coded by MPEG video 
format, bitrate represents a bitrate of the elementary data stream and the frequency represents a clock frequency 
of the virtual buffer. 

4. The method according to claim 13, further comprising the step of detecting a VBV delay (next_vbv_delay) of a 
picture to be inserted next, after a last picture in the first elementary data stream, by the following equation: 

next_vbv_delay = last_vbv_delay_(image_size/bitrate) + ADTS 

wherein last_vbv_delay represents a VBV delay of a last picture of the first elementary data stream, image_size 
represents an image size of the last picture and ADTS represents an interval of Decode Time Stamp; and 

wherein the controlling step detects the first bit occupancy of the inserted elementary data stream in the 
virtual buffer based on the VBV delay (next_vbv_delay) as a VBV delay of a first picture of the inserted elementary 
data stream. 

5. The method according to claim 14, further comprising the step of detecting the bit occupancy 
(first_picture_decode_size) in the virtual buffer at the arbitrary splice point of the second elementary data stream 
by the following equation: 

first_picture_decode_size = (first_vbv_delay - ADTS) x bitrate 
+ picture_start_code + sequence_header 

wherein first_vbv_delay represents a VBV delay of a first picture in the second elementary data stream, 
picture_start_code represents a picture start code of the first picture of the second elementary data stream, and 
sequence_header represents a sequence header of the first picture of the second elementary data stream; and 

wherein the controlling step inserts the stuffing byte along with the last picture of the inserted elementary 
data stream in accordance with the bit occupancy (first_picture_decode_size) so that the last bit occupancy of the 
inserted elementary data stream in the virtual buffer equals the bit occupancy (first_picture_decode_size). 

6. The method according to claim 11, further comprising the steps of detecting the bit occupancy (first 
picture_decode_size) in the virtual buffer at the arbitrary splice point of the second elementary data stream by the 
following equation: 

first picture_decode_size = (first_vbv_delay -ADTS) x bitrate 
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+ picture_start_code + sequence-header 

wherein first_vbv_delay represents a VBV delay of a first picture in the second elementary data stream, picture 
5 start_code represents a picture start code of the first picture of the second elementary data stream, 

sequence_header represents a sequence header of the first picture of the second elementary data stream and 
ADTS represents an interval of Decode Time Stamp; and 

wherein a stuffing byte is inserted along with the last picture of the inserted elementary data stream in ac- 
cordance with the bit occupancy (first_picture_decode_size) so that the last bit occupancy of the inserted elemen- 
10 tary data stream in the virtual buffer equals the bit occupancy (first_picture_decode_size). 

17. The method according to claim 11 , further comprising the steps of: 

detecting a VBV delay (next_vbv_delay) of a picture to be inserted next to a selected picture in the inserted 
is elementary data stream by the following equation: 

next_vbv_delay = vbv delay_(image_size/bitrate) + ADTS 

20 wherein vbv_delay represents VBV delay of the selected picture of the inserted elementary data stream, 

image_size represents an image size of the selected picture and ADTS represents an interval ol Decode Time 
Stamp; and 

inserting a stuffing byte corresponding to a data amount that the bit occupancy exceeds the virtual buffer along 
with the selected picture when the bit occupancy of the picture to be inserted next to the selected picture 
2S exceeds the virtual buffer 

18. The method according to claim 11 , wherein the inserted elementary data stream includes at least one intra-coded 
picture. 

30 19. The method according to claim 18, wherein the at least one intra-coded picture is a black picture. 

20. The method according to claim 18, wherein the inserted elementary data stream includes at least one repeat 
picture following the intra-coded picture, the repeat picture repeatedly displaying a previously displayed picture. 

35 21. The method according to claim 20, further comprising the step of: 

inserting at least one repeat picture until the last bit occupancy of the inserted elementary data stream in the 
virtual buffer exceeds the bit occupancy in the virtual buffer at the arbitrary splice point of the second elementary 
data stream; and 

40 inserting the pseudo data into the spliced elementary data stream thereafter. 

22. The method according to claim 12, wherein the inserted elementary data stream includes at least one of said 
repeat pictures, the repeat picture repeatedly displaying a previously displayed picture. 

45 23. The method according to claim 22, wherein the controlling step inserts the number of repeat pictures until the last 
bit occupancy of the inserted elementary data stream in the virtual buffer exceeds the bit occupancy in the virtual 
buffer at the arbitrary splice point of the second elementary data stream and then inserts a stuffing byte into the 
spliced elementary data stream. 

50 24. The method according to claim 15, wherein the inserted elementary data stream includes at least one repeat 
picture following at least one intra-coded picture, the repeat picture repeatedly displaying a proceeding picture. 

25. Apparatus for splicing a first elementary data stream containing a first compressed picture data and a second 
elementary data stream containing a second compressed picture data to generate a spliced elementary data 
55 stream containing a spliced continuous compressed picture data, said spliced elementary data stream being gen- 

erated by inserting an inserted elementary data stream between an arbitrary splice point of the first elementary 
data stream and an arbitrary splice point of the second elementary data stream, said apparatus comprising: 
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a first detector for detecting a bit occupancy in a virtual buffer at an arbitrary splice point of said first elementary 
data stream, said virtual buffer for controlling an amount of generated code data; 

a second detector for detecting a bit occupancy in said virtual buffer at an arbitrary splice point of said second 
elementary data stream; 

a controller for controlling said inserted elementary data stream so that a first bit occupancy of the inserted 
elementary data stream in said virtual buffer equals the bit occupancy in said virtual buffer at said arbitrary 
splice point of the first elementary data stream, and a last bit occupancy of the inserted elementary data stream 
in said virtual buffer equals the bit occupancy in said virtual buffer at said arbitrary splice point of the second 
elementary data stream; and 

a multiplexer for inserting the inserted elementary data stream between the arbitrary splice point of the first 
elementary data stream and the arbitrary splice point of the second elementary data stream to generate the 
spliced elementary data stream. 

26. The apparatus according to claim 25, wherein the inserted elementary data stream comprises a predetermined 
compressed picture data and pseudo data. 

27. The apparatus according to claim 25, wherein the virtual buffer exhibits a predetermined size; and the controller 
determines when the bit occupancy of the inserted elementary data stream in the virtual buffer exceeds the pre- 
determined size of the virtual buffer, and inserts pseudo data to have a size larger than the data amount that the 
inserted elementary data stream exceeds the predetermined size of the virtual buffer. 

28. The apparatus according to claim 25, wherein the inserted elementary data stream includes at least one intra- 
coded picture. 

29. The apparatus according to claim 28, wherein the at least one intra-coded picture is a black picture. 

30. The apparatus according to claim 28, wherein the inserted elementary data stream includes at least one repeat 
picture following the intra-coded picture, the repeat picture repeatedly displaying a proceeding picture. 

31. The apparatus according to claim 30, wherein the said controller inserts the at least one repeat picture until the 
last bit occupancy of the inserted elementary data stream in the virtual buffer exceeds the bit occupancy in the 
virtual buffer at the arbitrary splice point of the second elementary data stream and then inserts the pseudo data 
into the spliced elementary data stream. 

32. The apparatus according to claim 25, wherein the inserted elementary data stream includes at least one repeat 
picture, the repeat picture repeatedly displaying a previously displayed picture. 

33. The apparatus according to claim 32, wherein said controller inserts at least one repeat picture until the last bit 
occupancy of the inserted elementary data stream in the virtual buffer exceeds the bit occupancy in the virtual 
buffer at the arbitrary splice point of the second elementary data stream, and then inserts the pseudo data into the 
spliced elementary data stream. 

34. The apparatus according to claim 25, wherein the first and second elementary data streams and the inserted 
elementary data stream are coded data according to MPEG video format, respectively 

35. The apparatus according to claim 34, wherein the inserted elementary data stream comprises a predetermined 
coded picture data coded by MPEG video format and a stuffing byte defined in MPEG video format. 

36. The apparatus according to claim 35, wherein the controller detects the bit occupancy in the virtual buffer (VBV 
buffer) by the following equation: 

bit occupancy = VBV_delay x (bitrate/frequency), and 

wherein VBV delay represents a VBV delay of a picture in an elementary data stream coded by MPEG video 
format, bitrate represents a bit rate of the elementary data stream, and the frequency represents a clock frequency 
of the virtual buffer. 
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37. The apparatus according to claim 36, further comprising a detector for detecting a VBV delay (next _vbv_delay) 
of a picture to be inserted next after a last picture in the first elementary data stream, by the following equation: 

next_vbv_delay=last_vbv_delay_(image_size/bitrate)+ ADTS, 

wherein last_vbv_delay represents a VBV delay of a last picture of the first elementary data stream, 
image_size represents an image size of the last picture and ADTS represents an interval of Decode Time Stamp; 
and 

wherein the controller detects the first bit occupancy of the inserted elementary data stream in the virtual 
buffer based on the VBV delay (next_vbv_delay) as a stream coded by MPEG video format, the bitrate representing 
a bit rate and the frequency representing a clock frequency for representing the VBV delay (next_VBV_delay). 

38. The apparatus according to claim 37, further comprising a detector for detecting a VBV delay (next_vbv_delay) of 
a picture to be inserted next, after a last picture in the first elementary data stream by the following equation: 

next_vbv_delay = last_vbv_delay_(image_size/bitrate) + ADTS, 

wherein last_vbv_delay represents a VBV delay of a last picture of the first elementary data stream, 
image_size represents an image size of the last picture of the first elementary data stream and ADTS represents 
an interval of Decode Time Stamp; and 

wherein the controller detects the first bit occupancy of the inserted elementary data stream in the virtual 
buffer based on the VBV delay (next_vbv_delay) of a first picture of the inserted elementary data stream. 

39. The apparatus according to claim 38, further comprising: 

a second detector for detectingthe bit occupancy (first_picture_decode_size)ln the virtual buffer at the arbitrary 
splice point of the second elementary data stream by the following equation: 

first_picture_decode_size = (first_vbv_delay_ADTS) x bitrate 
+ picture_start_code + sequence_header, 

wherein first_vbv_delay represents a VBV delay of a first picture in the second elementary data stream, 
picture_start_code represents a picture start code of the last picture of the second elementary data stream, 
sequence_header represents the sequence header of the first picture of the second elementary data stream 
and ADTS represents an interval of Decode Time Stamp; and 

wherein the controller inserts a stuffing byte along with the last picture of the inserted elementary data stream 
in accordance with the bit occupancy 

(first _picture_decode_size) so that the last bit occupancy of the inserted elementary data stream in the virtual 
buffer equals the bit occupancy (first_picture_decode_size). 

40. The apparatus according to claim 39, 

wherein the controller detects a VBV delay (next_vbv_delay) of a picture to be inserted next to a selected 
picture in the inserted elementary data stream by the following equation: 

next_vbv_delay = vbv_delay_( image size/bitrate) + ADTS, 

wherein vbv_delay represents VBV delay of the certain picture of the inserted elementary data stream, 
image.size represents an image size of the certain VBV delay of a first picture of the inserted elementary data 
stream and ADTS represents an interval of Decode Time Stamp. 

41 . The apparatus according to claim 37, 

wherein the controller detects the bit occupancy (first picture_decode_size) in the virtual buffer (VBV buffer) 
at the arbitrary splice point of the second elementary data stream by the following equation: 
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first_picture_decode_size = (first_vbv_delay -ADTS) x bitrate 
+ picture_start_code + sequence_header, 

wherein first_vbv_delay represents a VBV delay of a first picture in the second elementary data stream, 
picture_start_code represents a picture start code of the first picture of the second elementary data stream, se- 
quence header represents sequence header of the first picture of the second elementary data stream and ADTS 
represents an interval of Decode Time Stamp; and 

wherein the controller inserts a stuffing byte along with the last picture of the inserted elementary data stream 
in accordance with the bit occupancy (first_picture_decode_size) so that the last bit occupancy of the inserted 
elementary data stream in the virtual buffer (VBV buffer) is equal to the bit occupancy (first_picture_decode size). 

42. The apparatus according to claim 41 , 

wherein the controller detects a VBV delay (next_vbv_delay) of a picture to be inserted next to a selected 
picture in the inserted elementary data stream by the following equation: 

next_vbv_delay = vbv_de!ay_(image_size/ bitrate) + ADTS 

wherein vbv_delay represents VBV delay of the selected picture of the inserted elementary data stream, 
image_size represents an image size of the selected picture and ADTS represents an interval of Decode Time 
Stamp; and 

wherein the controller inserts a stuffing byte corresponding to a data amount that the bit occupancy exceeds 
the virtual buffer along with the selected picture when the bit occupancy of the picture to be inserted next to the 
selected picture exceeds the virtual buffer. 

43. The apparatus according to claim 35, wherein the controller detects the bit occupancy (first_picture_decode_size) 
in the virtual buffer at the arbitrary splice point of the second elementary data stream by the following equation: 

first_picture_decode_size = (f irst_vbv_delay - ADTS) x bitrate 
+ picture_stari_code + sequencejieader 

wherein first_vbv_delay represents a VBV delay of a first picture in the second elementary data stream, 
picture_start_code represents a picture picture start code of the first picture of the second elementary data stream, 
sequence_header represents a sequence header of the first picture of the second elementary data stream and 
ADTS represents an interval of Decode Time Stamp; and 

wherein the controller inserts a stuffing byte along with the last picture of the inserted elementary data stream 
in accordance with the bit occupancy (first_picture_decode_size) so that the last bit occupancy of the inserted 
elementary data stream in the virtual buffer is equal to the bit occupancy (first_picture_decode_size). 

44. The apparatus according to claim 35, 

wherein the controller detects a VBV delay (next_vbv_delay) of a picture to be inserted next, after a selected 
picture in the inserted elementary data stream by the following equation: 

next_vbv_delay = vbv_delay_(image_size/ bitrate) + ADTS 

wherein vbv_delay represents VBV delay of the certain picture of the inserted elementary data stream, image size 
represents an image_size of the selected picture and ADTS represents an interval of Decode Time Stamp; and 

wherein the controller inserts a stuffing byte corresponding to a data amount that the bit occupancy exceeds 
the virtual buffer along with the selected picture when the bit occupancy of the picture to be inserted next, after 
the selected picture exceeds the virtual buffer. 

45. The apparatus according to claim 35, wherein the inserted elementary data stream includes at least one intra- 
coded picture. 
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46. The apparatus according to claim 45, wherein the at least one intra-coded picture is a black picture. 

47. The apparatus according to claim 45, wherein the inserted elementary data stream includes a number of repeat 
pictures following the intra-coded picture, the repeat picture repeatedly displaying a previously displayed picture. 

5 

48. The apparatus according to claim 47, wherein the controller inserts the number of repeat pictures until the last bit 
occupancy of the inserted elementary data stream in the virtual buffer exceeds the bit occupancy in the virtual 
buffer at the arbitrary splice point of the second elementary data stream, and then to insert the pseudo data into 
the spliced elementary data stream. 

10 

49. The apparatus according to claim 35, wherein the inserted elementary data stream includes a number of repeat 
pictures, the repeat picture repeatedly displaying a previously displayed picture. 

50. The apparatus according to claim 49, wherein the controller inserts the number of repeat pictures until the last bit 
75 occupancy of the inserted elementary data stream in the virtual buffer exceeds the bit occupancy in the virtual 

buffer at the arbitrary splice point of the second elementary data stream and then inserts a stuffing byte into the 
spliced elementary data stream. 

51. The apparatus according to claim 42, wherein the inserted elementary data stream includes a number of repeat 
20 pictures following at least one intra-coded picture, the repeat picture repeatedly displaying a previously displayed 

picture. 

52. A recording medium having a spliced elementary data stream representing a spliced continuous compressed pic- 
ture data recorded therein, said spliced elementary data stream being generated by splicing a first elementary 

25 data stream containing a first compressed picture data and a second elementary data stream containing a second 

compressed picture data, and by inserting an inserted elementary data stream between an arbitrary splice point 
of the first elementary data stream and an arbitrary splice point of the second elementary data stream, said spliced 
elementary data stream being prepared by the steps of: 

so detecting a bit occupancy in a virtual buffer at the arbitrary splice point of said first elementary data stream, 

the virtual buffer being provided for controlling an amount of code data which is generated during decoding; 
detecting a bit occupancy in the virtual buffer at the arbitrary splice point of the second elementary data stream; 
and 

controlling the inserted elementary data stream so that a first bit occupancy of the inserted elementary data 
35 stream in the virtual buffer equals the bit occupancy in a virtual buffer at the arbitrary splice point of the first 

elementary data stream and a last bit occupancy of the inserted elementary data stream in the virtual buffer 
equals the bit occupancy in the virtual buffer at the arbitrary splice point of the second elementary data stream. 

53. A recording medium producible by a receiving device, the record medium having a recorded signal recorded ther- 
40 eon, the recorded signal including a spliced elementary data stream containing a spliced continuous compressed 

picture data, the spliced continuous picture data being generated by splicing a first elementary data stream con- 
taining first compressed picture data and a second elementary data stream containing a second compressed 
picture data stream, and by inserting an inserted elementary data stream containing a data stream to be inserted 
between an arbitrary splice point of the first elementary data stream and an arbitrary splice point of the second 
45 elementary data stream, said spliced elementary data stream being prepared by the steps of: 

detecting a bit occupancy in a virtual buffer at the arbitrary splice point of the first elementary data stream, the 
virtual buffer being provided for controlling an amount of code which is generated during decoding; 
detecting a bit occupancy in the virtual buffer at the arbitrary splice point of the second elementary data stream; 
so and 

controlling the inserted elementary data stream so that a first bit occupancy of the inserted elementary data 
stream in the virtual buffer equals the bit occupancy in the virtual buffer at the arbitrary splice point of the first 
elementary data stream and a last bit occupancy of the inserted elementary data stream in the virtual buffer 
equals the bit occupancy in the virtual buffer at the arbitrary splice point of the second elementary data stream. 

55 
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